Over the past few years, the object recognition community has taken on the challenge of developing systems that learn to recognize hundreds of object classes from a few examples per class. The standard data sets used for benchmarking these systems contained, in average, fewer than two hundred images per class (as opposed to sets of thousands of meticulously segmented object images that were used in earlier work on object detection). Benchmarking on such small data sets is inherently problematic. The test results cannot be generalized and can be misleading.
There have been efforts of building larger databases of manually annotated, natural images. However, the somewhat arbitrary selection of images and the missing ground truth make it difficult to systematically analyze specific properties of object recognition systems, such as invariance to pose, scale, position, and illumination. A database for shape-based object recognition which addresses these issues is NORB from the Courant Institute at New York University. Pictures of objects were taken with consideration of viewpoint and illumination. The images were synthetically altered to add more image variations, such as object rotation, background, and “distractors”.
Taking the idea of controlling the image generation one step further takes us to fully synthetic images rendered from realistic three-dimensional (3D) computer graphics models. Some view-based object recognition systems have been trained and evaluated on synthetic images. At least one face recognition system and one object recognition system have been trained on views of 3D models and tested on real images. 3D models have also been used in a generative approach to object recognition in which rendering parameter values are optimized such that the synthetic image best matches a given photographic image. To avoid getting trapped in local minima, this analysis-through-synthesis approach requires a good initial estimate of the rendering parameter values, making it unsuited to many object recognition/detection tasks.